Common Reference

tpm cert

Version availability

Available in 7.1.0 and later.

The tpm cert command is designed to streamline the generation of Tungsten-specific security files for use by the tpm install and tpm update commands.

Usage

tpm cert [action] [typeSpec] [args]

Be Aware

The tpm cert command minimizes the complexity of handling certificates but cannot remove it entirely. For the best results, contact Continuent Support and get help from the experts before using this command.

By default, the tpm install command will take care of all security files in new installations (upgrades preserve existing) when disable-security-controls=false or when the option is omitted when installing v7.0.0 or later.

You may need to provide your own certificates, or handle enabling security at a later time and install with disable-security-controls=true. If so, tpm cert is for you.

Cert SOURCE files are in the "certsdir" $CONTINUENT_ROOT/generated/
Cert RUNNING files are in the "security directory" $CONTINUENT_ROOT/share/

By default, all commands use source files located in "certsdir". To read running files in the "security directory", use --running (or -r).

To use custom values/paths, create a file called tungsten.env in the "security directory" and populate the variables as needed.

important

There is no way to rotate security files with zero downtime. This is a database server limitation because once the first database server process is restarted with the new certs, the cluster will not be able to communicate with it. The same goes for the tpm update step if done before the database server process restarts. Once all database servers are using the new certs and all cluster nodes have been updated, everything will be able to communicate properly and the operation will be done.

Limitation

In versions prior to 7.1.3, tpm cert cannot be run if the Tungsten software is not yet installed. Upgrade to 7.1.3+ to make use of this enhanced behavior.

Read-Only Actions

Option	Description	Version
`aliases` Aliases: `al`	Display alias names from one or more files.
`ask` Aliases: `as`	Display various information.
`cat` Aliases: `ca`	Display key files.
`diff` Aliases: `d`	Compare running files with generated files.
`example` Aliases: `ex`	Display example files.
`help` Aliases: `h`	Display short help text.
`info` Aliases: `in`	Display metadata about a security file as JSON.
`list` Aliases: `li`	Show the contents of a security file.
`ls`	List a directory.

Write Actions

Option	Description	Version
`add` Aliases: `adimportim`	Add one or more typeSpecs into another.
`backup` Aliases: `ba`	Backup one or more key directories and files.
`changepass` Aliases: `chcp`	Change the storepass for one or more files.
`clean` Aliases: `cl`	Delete all files in a directory.
`copy` Aliases: `co`	Copy one or more key directories or files to other node(s) in the cluster, either before or after install.	7.1.3
`gen` Aliases: `gcreatecr`	Generate various security files.
`remove` Aliases: `remrm`	Delete a specific alias from a security file.
`rotate` Aliases: `roswapsw`	Replace an existing entry with one from another file.
`update`	Update the specified typeSpec.	7.2.0
`vi` Aliases: `v`	Edit the file.

Arguments

Option	Description	Version
`--count` Aliases: `-c`	Display an integer count of aliases found instead of the actual aliases.
`--debug` Aliases: `-d`	Displays debug-level status messages.
`--dir`	Specify the target directory to store files in
`--dryrun` Aliases: `-n`	Do not execute the command, display what would be done instead.
`--extra` Aliases: `-x`	Display the command to be run before executing, and other additional information when available.
`--generated`
`--help` Aliases: `-h`	Displays a help message.
`--i-am-sure`	Confirm you want the DESTRUCTIVE operation (delete/rotate) to proceed without an interactive pause.
`--info` Aliases: `-i`	Displays info-level status messages.
`--livetls`	Use the running `tungsten_tls_keystore.jks` in `$CONTINUENT_ROOT/share/`. You may not use `--tls` and `--livetls` together.
`--long` Aliases: `-l`	Display verbose output in keytool and openssl and other areas.
`--mysqldir`	Specify the target directory to store MySQL-specific files in.
`--quiet` Aliases: `-q`	Hides status output whenever possible.
`--running` Aliases: `-r`	Use the running files from `$CONTINUENT_ROOT/share/` instead of the certs source directory `$CONTINUENT_ROOT/generated/`.
`--tls`	Specify a source TLS typeSpec (either `tls_keystore` or `TLS_FILE`).
`-v`	Displays verbose-level status messages.

To see more detailed help on each action, you can use the following commands:

shell> tpm cert h [action]

`tpm cert [typeSpec]`, Defined

A typeSpec is a case-sensitive, unique string that identifies a key security file, possibly located in different subdirectories. Examples include:
- keystore = tungsten_keystore.jks
- connector_keystore = tungsten_connector_keystore.jks

[typeSpec] can be either a single string or a comma-separated list with no spaces, for example:

shell> tpm cert info connector_truststore
shell> tpm cert aliases jgroups_keystore,tls_keystore

Use tpm cert help typespec to see the standard typeSpec
Use tpm cert help [action] to see the typeSpec for that action
Different [action]s have different typeSpecs
Some typeSpecs are groups of other typeSpecs
There are three classes of typeSpec:
1. Pre-Defined Source Tungsten-specific files
- These files are located in $CONTINUENT_ROOT/generated/ and are populated by tpm cert gen
1. Pre-Defined "Running" Tungsten & MySQL-specific files
- Tungsten-specific files are located in $CONTINUENT_ROOT/share/, and are populated by tpm install or
tpm update
- MySQL-specific files are located in the MySQL datadir by default, and can be populated with tpm cert gen mysqlcerts
- The Tungsten running files are accessed by adding --running (or -r) on the command line while using the same typeSpec - for example,
  - tpm cert info ct displays $CONTINUENT_ROOT/generated/connector_truststore.ts
  - tpm cert info ct -r displays $CONTINUENT_ROOT/share/connector_truststore.ts
1. User-Defined Source and Target Files
- These files are typically located in $BASE_DIR/ and are configured via $CONTINUENT_ROOT/share/tungsten.env. They are also populated by tpm cert gen
- Example BASE_DIR values, possibilities are endless, as long as the Tungsten OS user (usually tungsten) has write access to that directory or the ability to create that directory:
  - /etc/tungsten/secure
  - /var/lib/mysql
  - /home/tungsten/certs

[typeSpec] definitions

PRE-Defined RUNNING Tungsten-specific files

Option	Description	Product
`connector_keystore` Aliases: `cjck`	`tungsten_connector_truststore.ts`	CT
`connector_truststore` Aliases: `ct`	`tungsten_connector_truststore.ts`	CT
`jgroups_keystore` Aliases: `jg`	`tungsten_jgroups_keystore.jceks`	CT
`jmx_passwordstore` Aliases: `jm`	`jmxremote.access`	CTTR
`keystore` Aliases: `jkke`	`tungsten_keystore.jks`	CTTR
`thl_keystore` Aliases: `tk`	`tungsten_thl_keystore.jks`	CTTR
`thl_truststore` Aliases: `tt`	`tungsten_thl_truststore.ts`	CTTR
`tls_keystore` Aliases: `tl`	`tungsten_tls_keystore.jks`	CTTR
`tls_passwordstore` Aliases: `pw`	`passwords.store`	CTTR
`truststore` Aliases: `tstr`	`tungsten_truststore.ts`	CTTR

Access these via cli arg --running (or -r)
Located in $CONTINUENT_ROOT/share/, populated by tpm install or tpm update

PRE-Defined RUNNING Tungsten-specific MySQL files

Option	Description	Product
`mysqlca` Aliases: `mc`	`ca.pem`	CTTR
`mysqlcert` Aliases: `mt`	`client-cert.pem`	CTTR
`mysqlkey` Aliases: `mk`	`client-key.pem`	CTTR

Located in MySQL datadir by default, populated by tpm cert gen mysqlcerts

PRE-Defined SOURCE Tungsten-specific files

Option	Description	Product
`connector_keystore` Aliases: `cjck`	`tungsten_connector_truststore.ts`	CT
`connector_truststore` Aliases: `ct`	`tungsten_connector_truststore.ts`	CT
`jgroups_keystore` Aliases: `jg`	`tungsten_jgroups_keystore.jceks`	CT
`jmx_passwordstore` Aliases: `jm`	`jmxremote.access`	CTTR
`keystore` Aliases: `jkke`	`tungsten_keystore.jks`	CTTR
`mysqlp12` Aliases: `p1`	`client-cert.p12`	CTTR
`public`	tungsten_public,jks	CTTR
`thl_keystore` Aliases: `tk`	`tunsten_thl_keystore.jks`	CTTR
`thl_triststore` Aliases: `tt`	`tungsten_thl_truststore.ts`	CTTR
`tls_keystore` Aliases: `tl`	`tungsten_tls_keystore.jks`	CTTR
`tls_passwordstore` Aliases: `pw`	`passwords.store`	CTTR
`truststore` Aliases: `tstr`	`tungsten_truststore.ts`	CTTR

Located in $CONTINUENT_ROOT/generated/, populated by tpm cert gen

USER-Defined Source and Target Files

Option	Description	Product
`CA_PEM_FILE` Aliases: `CA`	.pem CA file generated by MySQL	CTTR
`CERT_PEM_FILE` Aliases: `CE`	.pem file generated from the .crt file	CTTR
`CJKS_FILE` Aliases: `CJ`	Tungsten Connector Keystore, i.e. `tungsten_connector_keystore.jks`	CT
`CRT_FILE` Aliases: `CR`	.crt file generated from the .pfx file	CTTR
`CTS_FILE` Aliases: `CT`	Tungsten Connector Truststore, i.e. `tungsten_connector_truststore.ts`	CT
`JGROUPS_FILE` Aliases: `JG`	Tungsten jGroups Keystore, i.e. `tungsten_jgroups_keystore.jceks`	CT
`JKS_FILE` Aliases: `JK`	Tungsten Keystore, i.e. `tungsten_keystore.jks`	CTTR
`JMX_FILE` Aliases: `JM`	Tungsten RMI/JMX Password Store, i.e. `jmxremote.access`	CTTR
`KEY_FILE` Aliases: `KEY`	.key and .key.encrypted generated from the .pfx file	CTTR
`P12_FILE` Aliases: `P1`	.p12 file generated from the .pem and .key files	CTTR
`PFX_FILE` Aliases: `PF`	Source .pfx file, i.e. `mysql.pfx`	CTTR
`PW_FILE` Aliases: `PW`	Tungsten TLS Password Store, i.e. `passwords.store`	CTTR
`TJKS_FILE` Aliases: `TJ`	Tungsten THL Keystore, i.e. `tungsten_thl_keystore.jks`	CTTR
`TLS_FILE` Aliases: `TL`	Tungsten TLS Keystore, i.e. `tungsten_tls_keystore.jks`	CTTR
`TS_FILE` Aliases: `TS`	Tungsten Truststore, i.e. `tungsten_truststore.ts`	CTTR
`TTS_FILE` Aliases: `TT`	Tungsten THL Truststore, i.e. `tungsten_thl_truststore.ts`	CTTR

Typically located in $BASE_DIR/ and configured via$CONTINUENT_ROOT/share/tungsten.env

Convenience tags

Option	Description	Version
`all` Aliases: `a`	Varies based on the following factors: If `tungsten.env` exists, then use the User-defined files from it, else use the pre-defined files from `$CONTINUENT_ROOT/generated`, or from `$CONTINUENT_ROOT/share` if `--running` is specified on the cli.
`batch` Aliases: `b`	Runs typeSpec defined in BATCH envvar, comma-separated.
`pfx`	Runs PFX_FILE,user
`tungsten` Aliases: `tu`	Runs pre-defined: tl,jg,jk,ts,cj,ct,tj,tt,pw,jm
`user`	Runs user-defined TL,JG,JK,TS,CJ,CT,TJ,TT,PW,JM

"Convenience tags" can be used in place of specific [typeSpec] options.

`[passwordSpec]` definitions

Used to specify passwords in a standard format for the openssl and keytool commands so that you do not need to remember the command-specific syntax, which is different for each command.

Password Specifications MUST be one of:

env:VARNAME - has to be pre-defined and exported as an EnvVar in the $CONTINUENT_ROOT/share/tungsten.env file.
run:typeSpec - available as part of the running Tungsten config for [typeSpec] of:
- keystore|ke
- truststore|ts
- connector_keystore|ck
- connector_truststore|ct
pass:yourPasswordHere - specify the actual password string and be sure to escape any special characters from the shell.

If the >PasswordSpec provided does not begin with env:, run: or pass: it will be rejected.

Getting Started - Basic Examples

Display each typeSpec and the list of aliases found in it:

shell> tpm cert aliases all
shell> tpm cert aliases all -c
shell> tpm cert aliases all -l -x

Display the file information as JSON, including SHA and Expiration date:
```
shell> tpm cert info tungsten
shell> tpm cert info tungsten -r
```

Use keytool or openssl to display the contents of the file:

shell> tpm cert list truststore
shell> tpm cert list truststore -l -x

Display an example of needed INI entries for custom work:
```
shell> tpm cert example ini
shell> tpm cert ex i
```
Edit the /etc/tungsten/tungsten.ini file using vi:
```
shell> tpm cert vi ini
shell> tpm cert v i
```

Display an example of needed tungsten.env entries for custom work:

shell> tpm cert example env
shell> tpm cert ex env 1
shell> tpm cert ex e 2

Generate a new $CONTINUENT_ROOT/share/tungsten.env file:
```
shell> tpm cert gen env 1
```
Edit the $CONTINUENT_ROOT/share/tungsten.env file using vi:
```
shell> tpm cert vi env
shell> tpm cert v e
```

Getting Started - Functional Database Cert Rotation Example

The philosophy here is that the cert rotation work is done on a single cluster node. We will call this the "work node".

The secret to getting certs to work with a cluster is to make sure that you copy all of the MySQL database certs and Tungsten security files from the work node to the rest of the nodes at the correct point in the process. If all the cluster and database cert files are not the same across all nodes, the cluster will fail.

In the following functional example, we demonstrate the actual steps to rotate the database certs in a standalone 5-node Tungsten cluster.

When doing a database cert rotation, multiple files must be regenerated:

Five (5) required MySQL security files (may be more created than needed):
1. ca.pem
2. client-cert.pem
3. client-key.pem
4. server-cert.pem
5. server-key.pem
One (1) .p12 file to represent the database client cert:
1. client-cert.p12
Four (4) Tungsten-specific files:
1. tungsten_keystore.jks
2. tungsten_truststore.ts
3. tungsten_truststore.ts
4. tungsten_connector_truststore.ts

Next, confirm that both MySQL and Tungsten are configured to use the proper files:

/etc/my.cnf entries:

[mysqld]
ssl-ca=/etc/mysql/certs/ca.pem
ssl-cert=/etc/mysql/certs/server-cert.pem
ssl-key=/etc/mysql/certs/server-key.pem
require_secure_transport=ON

important

This example assumes that the database certs are located in /etc/mysql/certs on all cluster database nodes.

/etc/tungsten/tungsten.ini entries:

datasource-mysql-ssl-ca=/etc/mysql/certs/ca.pem
datasource-mysql-ssl-cert=/etc/mysql/certs/client-cert.pem
datasource-mysql-ssl-key=/etc/mysql/certs/client-key.pem

important

This example assumes that no /opt/continuent/share/tungsten.env file exists.

>> FUNCTIONAL PROCEDURE EXAMPLE STARTS HERE

On a single node (db1) as the tungsten OS user
- Backup the old database certs from /etc/mysql/certs/ to /opt/continuent/backups/:
```
shell> tpm cert backup mysql
```
- Clean out any old database certs so new certs are generated:
```
shell> sudo rm /etc/mysql/certs/*.pem
```
- Verify all old database certs are gone:
```
shell> ls -l /etc/mysql/certs/*.pem
```
- Generate new mysql certs:
```
  shell> tpm cert gen mysqlcerts -x
```
- Set proper ownership and permissions for Tungsten access:
```
shell> sudo chown -R mysql: /etc/mysql/certs/
shell> sudo chmod -R g+r /etc/mysql/certs/
```
- Verify new database certs:
```
shell> ls -l /etc/mysql/certs/*.pem
```
- Copy new database certs to all other database nodes:
```
shell> for i in 2 3 4 5; do 
sudo rsync -avc --delete /etc/mysql/certs/ db$i:/etc/mysql/certs/; 
done
```
- Backup any previously-generated cluster certs to /opt/continuent/backups/:
```
shell> tpm cert backup gen
```
- Backup the running cluster certs to /opt/continuent/backups/:
```
shell> tpm cert backup share
```
- Regenerate the MySQL .p12 file and the needed Tungsten security files, using --livetls to specify the running TLS cert file (/opt/continuent/share/tungsten_tls_keystore.jks):
```
shell> tpm cert gen p12,ke,ts,ck,ct --livetls
```
- Examine the new files:
```
shell> tpm cert info p12,ke,ts,ck,ct
```
- Copy new files to ALL other nodes:
```
shell> tpm copy --gen
```
  Or:
```
shell> for i in 2 3 4 5; do 
rsync -avc --delete /opt/continuent/generated/ db$i:/opt/continuent/generated/; 
done
```
- If running Tungsten Clustering, set the cluster policy to MAINTENANCE
```
shell> tpm policy -m
```
On all other nodes nodes as the tungsten OS user
- Stop the processes:
```
shell> stopall
```
- Restart the database server process:
```
shell> sudo systemctl restart mysqld
```
  Or:
```
shell> sudo service mysqld restart
```
- Identify the Tungsten software staging directory:
```
shell> cd `tpm query staging| cut -d: -f2`
```
- Update the Tungsten software to use the new certs, which will restart all Tungsten processes:
```
shell> tools/tpm update -i --replace-release
```
- When all updates have completed, start the software:
```
shell> startall
```
- Test connectivity
- If running Tungsten Clustering, and the above test succeeds, set the cluster policy to AUTOMATIC:
```
shell> tpm policy -a
```

Getting Started - Conversion to Custom-Generated Security Files Example

In the following example, we take an existing cluster that was installed using Tungsten-self-generated security files and convert it to use custom-generated security files.

The basic security file conversion steps are:

Ensure three tpm config options exist and point to the correct files:

shell> tpm query config | grep datasource_mysql_ssl

datasource_mysql_ssl_ca (tpm option: datasource-mysql-ssl-ca)
datasource_mysql_ssl_cert (tpm option: datasource-mysql-ssl-cert)
datasource_mysql_ssl_key (tpm option: datasource-mysql-ssl-key)

Generate all standard security files and place into [certsdir] on ONE node only
```
shell> tpm cert gen all
```
Display new files info as json for standard cert files in [certsdir]
```
shell> tpm cert info all
```
Copy new files to all other cluster nodes
```
shell> tpm copy --gen
```
Display example tungsten.ini contents
```
shell> tpm cert example ini
```
Add those lines to the /etc/tungsten/tungsten.ini on ALL cluster nodes
```
shell> tpm cert vi ini
```
If running Tungsten Clustering, place the cluster into AUTOMATIC mode
```
shell> tpm policy -m
```
Display the directory that the software was installed from:
```
shell> tpm query staging
```
Update the tungsten software using tpm to use the new security files in [certsdir] which will restart the Tungsten processes:
```
shell> cd [staging_dir_from_above]
shell> tools/tpm update --replace-release
```
If running Tungsten Clustering, once all cluster updates are done, return the cluster to AUTOMATIC mode
```
shell> tpm policy -a
```

Getting Started - Advanced Example

You may want to provide your own certificates, or have installed with disable-security-controls=true, and now wish to enable security. If so, tpm cert is for you.

In the following advanced example, we will rotate the database certs using a source .pfx file.

--- Summary ---

Populate the tungsten.env file
Generate the security files defined in tungsten.env
Add new options to the /etc/tungsten/tungsten.ini to match
Update the software using the new security settings

--- Details ---

Displays example tungsten.env contents
```
shell> tpm cert example env
```
Create a new $CONTINUENT_ROOT/share/tungsten.env file, which defaults to example id 1:
```
shell> tpm cert gen env 2
```

Run vi $CONTINUENT_ROOT/share/tungsten.env

shell> tpm cert vi env
export BASE_DIR=/etc/tungsten/secure
export BATCH="pfx2p12,JK,TS,CJ,CT"

Display variables set in $CONTINUENT_ROOT/share/tungsten.env
```
shell> tpm cert ask env
```
Displays example tungsten.ini contents
```
shell> tpm cert example ini
```

Run vi /etc/tungsten/tungsten.ini

shell> tpm cert vi ini
java-keystore-path=/etc/tungsten/secure/tungsten_keystore.jks
java-truststore-path=/etc/tungsten/secure/tungsten_truststore.ts
java-connector-keystore-path=/etc/tungsten/secure/tungsten_connector_keystore.jks
java-connector-truststore-path=/etc/tungsten/secure/tungsten_connector_truststore.ts

Generate all cert files in the BATCH envvar defined in the tungsten.env file:
```
shell> tpm cert gen batch --livetls -x
```
Display info as json all cert files in the BATCH envvar defined in the tungsten.env file:
```
shell> tpm cert info P12,JK,TS,CJ,CT
```
Display the extracted package staging directory that the software was installed from:
```
shell> tpm query staging
```

Update the software to use the new cert files in [certsdir]:

shell> cd {staging_dir}
shell> tools/tpm update --replace-release

tpm cert [typeSpec], Defined​

[typeSpec] definitions​

[passwordSpec] definitions​

Getting Started - Basic Examples​

Getting Started - Functional Database Cert Rotation Example​

Getting Started - Conversion to Custom-Generated Security Files Example​

Getting Started - Advanced Example​

`tpm cert [typeSpec]`, Defined

[typeSpec] definitions

`[passwordSpec]` definitions

Getting Started - Basic Examples

Getting Started - Functional Database Cert Rotation Example

Getting Started - Conversion to Custom-Generated Security Files Example

Getting Started - Advanced Example